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ABSTRACT 



The State Children’s Health Insurance Program (SCHIP) in 
1997 signaled a major increase in time, energy, creativity, and money devoted 
to enrolling eligible children in health insurance programs. Child advocates 
and others have focused on developing outreach and enrollment strategies to 
bring the benefits of these new SCHIP programs to children and their 
families. Noting that knowing whether these strategies are working is as 
urgent as the strategies themselves, this issue brief discusses the role of 
participatory evaluation in keeping a clear focus on the purpose of the 
project and its learning objects in order to develop a program which can be 
evaluated for effectiveness, not only at the project's completion, but while 
the project is underway. Every project component- -planning the activities, 
identifying the participants, developing the training, deciding on the data 
to be collected, and designing the tracking forms- -needs to be measured 
against these questions: Why are we doing this project? and What do we want 
to know when the project is over? An interview with Pat Redmond, the health 
director at the Philadelphia Citizens for Children and Youth, is presented to 
illustrate how participatory evaluation leads to a project that can be 
evaluated for effectiveness. (KB) 
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T he State Children’s Health Insurance Program 
(SCHIP) was established as a part of the Balanced 
Budget Act of 1997 (BBA). Immediately following 
passage of the law, child advocates began working 
to shape state SCHIP program designs which 
would benefit the maximum number of children. They 
advocated generous eligibility levels and comprehensive 
benefits. The BBA required that, prior to enrollment in a 
state SCHIP program,' a child be screened for eligibility 
in Medicaid and, if Medicaid-eligible, be enrolled in 
Medicaid rather than SCHIP. As a result, SCHIP outreach 
and enrollment efforts have become SCHIP/Medicaid 
outreach and enrollment programs instead of programs 
de\'oted to SCHIP alone, building on existing Medicaid 
ouu-each work where such programs existed, and stimu- 
ladng broad new efforts elsewhere. 

Child advocates have become hea\hly involved in out- 
reach and enrollment work. In eleven states, child advo- 
cacy organizations serve as the lead grantee in the 
Robert Wood Johnson-funded CoveringKids initiative. 
Numerous other state-, city-, and community-based 
child advocacy organizations are included in outreach 
and enrollment, either as CoveringKids pilot sites, com- 
mittee members, or partnering organizations, or 
through other outreach and enrollment work. Many 
child advocates have adopted the label “child health 
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T he establishment of the state children’s health 
insurance program (SCHIP) as part of the 
Balanced Budget Act of 1997 (BBA), signaled a major 
increase in time, energy, creativity, and money devoted 
to enrolling eligible children in health insurance pro- 
grams. Child advocates and others energetically tackled 
the challenge of crafting state-level SCHIP programs 
and undertaking the outreach and enrollment needed 
to bring the benefits of these new SCHIP programs to 
children and their parents. 



insurance outreach and enrollment” for SCHIP/Medicaid 
outreach and enrollment campaigns to emphasize their 
goal of gaining health care coverage for all children, regard- 
less of the specific type of health care program entailed. 

Other entities have been active in outreach and enrollment 
as well. States have undertaken a wide range of activities,^ 
including promotional materials, radio and television pro- 
motion, toll-free hotlines, web sites, outstationing of eligibil- 
ity workers, and more. In some states, SCHIP and Medicaid 
have been given a single name and use common marketing 
materials, including a single application form for both. 
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Some states have established partner- 
ships or advisory committees includ- 
ing advocates, parents, providers, and 
community-based organizations. 

The federal government has likewise 
been active. Under the slogan “Insure 
Kids Now,” it sponsored a public edu- 
cation campaign including a national 
1-800 number to increase public 
awareness of the new SCHIP pro- 
grams.’ Several federal agencies, 
including the Department of Health 
and Human Services, the Department 
of Education, the Department of 
Agriculture, and the Department of 
Justice have collaborated on highly 
visible joint projects related to SCHIP, 
and a variety of other federal agencies 
have undertaken some activity 
designed to increase enrollment in 
SCHIP. The federal government iden- 
tified incorporation of outreach and 
enrollment activities in schools across 
the country as a priority activity, and 
sponsored an “Education Summit” to 
further this initiative. 

Foundations and other national and 
local groups with an interest in pro- 
moting child health have stepped up 
to the plate wath funding to support a 
variety of programs. Notable among 
these is the Robert Wood Johnson 
Foundation’s $47 million dollar, 
three-year CoveringKids initiative’ 
noted above, with funded programs 
up and running in 49 of the 50 states 
as well as the District of Columbia. 
This initiative supports outreach and 
enrollment programs for both SCHIP 
and Medicaid, with strong emphasis 
on the goals of simplification of appli- 
cation and enrollment and of coordi- 
nation with existing benefits coverage 
programs. Other national and local 
entities ranging from corporations to 



charitable nonprofit organizations are 
likewise involved, through public 
education efforts designed to increase 
awareness, through direct support of 
outreach and enrollment, or through 
a host of other activities. 



and the significant resources now 
engaged in efforts to do so, the ques- 
tion of how to assess outreach and 
enrollment programs to identify what 
works and what doesn’t is an obvious 
and significant one. 



Evaluating the Impact 
of Outreach and 
Enrollment Activities 

W hat are the results of all these 
activities? When the SCHIP 
program was established, 
proponents spoke of achiev- 
ing SCHIP enrollments of 
well over four million children. A sur- 
vey of states to determine SCHIP 
enrollment as of a set date, June 30, 
1999, showed that, on that date, 
1,979,450 children were enrolled in 
SCHIP programs.’ The most current 
report, based on data submitted to 
HCFA by the states for federal 
fiscal year 1999 (10/1/98- 
9/30/99), indicates that 
1,979,450 children 
were enrolled in 
SCHIP programs at 
some point during 
the year,® indicating 
that enrollment 
progress is being 
made. At the same 
time, reports of 
falling enrollments in 
Medicaid programs 
around the country, 
most related at least in 
part to improper or inad- 
equate state approaches to 
maintenance of Medicaid 
enrollment when families leave 
TANF, have given rise to great con- 
cerns in the child advocacy communi- 
ty.' Given the importance of increas- 
ing child health insurance coverage 
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The question of how to accomplish 
such assessment is not only impor- 
tant, but complex and difficult as 
well. Outreach and enrollment efforts 
occur on many levels - local, state, 
and national. They take many forms - 
targeted community presentations at 
churches, schools, and community 
centers; outstationing of outreach and 
eligibility workers in schools and 
child care centers; training of commu- 
nity leaders to lead local outreach pro- 
grams; outreach in community health 
centers and hospitals; use of paid 
application assistance or facilitated 
enrollment; public service 
announcements; campaigns 
publicizing SCHIP names 
and 1-800 numbers. It is 
difficult to disentangle 
the impact of one form 
of outreach from that 
of others. It is not 
possible to say with 
precision whether 
the first, third, or 
seventh contact 
which a parent has 
4 with information 
about the local SCHIP 
program is the contact 
which prompts that par- 
ent to initiate the applica- 
tion process, or what role 
national media or awareness 
efforts play in the mix. Further, 
because there are significant cultural 
differences among some of the groups 
with large proportions of children 
without health care coverage, it is 



Participatory 
evaluation proposes ’ 
that assessment of 
effectiveness can be 
achieved as a project is 
underway - if project plan- 
ning is done with the end 
in mind, if ongoing data 
collection is designed to pro- 
vide information needed to 
assess effectiveness, and 
if evaluation activities 
\ are structured into the 
project design. 



best copy available 









highly likely that methods which are 
effective with some groups are less so 
with others. 

Nonetheless, given the importance of 
identifying strategies that work and 
the significant quantities of resources 
involved, the challenge of evaluation 
must be addressed. Further, child 
advocates and others who are actively 
engaged in outreach and enrollment 
efforts do not want to defer evaluation 
of effectiveness until programs 
are over, as more traditional, formal 
models of evaluation would suggest. 
Rather, child advocates seek timely 
information on strategies which work 
so they can use their resources to best 
effect, both in their own outreach and 
enrollment efforts, and in administra- 
tive and legislative advocacy efforts in 
support of successful strategies. 

Participatory evaluation proposes that 
assessment of effectiveness can be 
achieved as a project is underway - if 
project planning is done with the end 
in mind, if ongoing data collection is 
designed to provide information needed 
to assess effectiveness, and if evaluation 
activities are structured into the project 
design. This Issue Brief presents an 
interview with Pat Redmond, Health 
Director at NACA member organization 
Philadelphia Citizens for Children and 
Youth (PCCY). PCCYs design of their 
Medicaid/CHIP enrollment fund project 
is a useful example of how “planning 
the beginning with the end in mind” 
leads to a project which can be evaluat- 
ed for effectiveness. PCCY is a pilot site 
for Pennsylvania’s CoveringKids initia- 
tive, managed by lead grantee 
Pennsylvania Parmerships for Children, 
also a NACA member organization. The 
enrollment fund is a CoveringKids proj- 
ect, which began in 1999. 



NACA: What is the enrollment 
fund strategy all about? 

Redmond: Let’s start with some 
context. As in other places, there 
are many uninsured children in 
Philadelphia and the surrounding 
counties in the region. But awareness 
of Medicaid and CHIP is proba- 
bly higher than in some parts 
of the country, for at least 
two reasons. One, there 
is a high child poverty 
rate in Philadelphia, 
and so Medicaid has 
been critically impor- 
tant for many years. 

At one point, almost 
60 percent of chil- 
dren in the city were 
enrolled in Medicaid, 
although this number 
has fallen since wel- 
fare reform. A second 
reason is that 
Pennsylvania has had a 
CHIP program since 1992 
“ our state’s program was 
one of the models for the 
national SCHIP program. The state, 
state-contracted insurance companies, 
and community groups have been pro- 
moting CHIP for many years. 

PCCY plays a key role on this issue: we 
brought together a Greater Philadelphia 
Child Health Insurance Coalition on 
child health issues many years ago, 
developed user-friendly outreach mate- 
rials and enrollment tools, and trained 
neighborhood groups and health care 
providers on the ins and outs of 
enrolling children. We’ve also taken an 
active role in advocating for systems 
that work well for families, and have 
been gratified to see real improvements 
take place in Pennsylvania. But there’s 
still a lot of work to be done. 



The CoveringKids project offered us 
the opportunity to try something new 
“ to experiment with a model that 
might work to engage organizations 
that care about children’s health, but 
haven’t been able to put real time and 
resources into enrolling kids. We and 
our coalition partners decided 
that an enrollment fund 
might work in this region. 

NACA: What do 
you mean by 
""enrollment fund”? 
Where did you 
get the idea? 

Redmond: We got 

the idea from 
California and other 
states with finders 
fees, application assis- 
tance, or similar pro- 
/ grams - there are a vari- 
ety of names. What they 
have in common is that 
they try to encourage out- 
reach and enrollment work by 
paying a set fee for submission of 
completed SCHIP and Medicaid appli- 
cations to individuals or groups who 
have helped parent organizations to 
complete those applications. 

NACA: Why did you cIkm>m 
this idea over others? 

Redmond: There is a lot of interest 
in children’s health insurance in this 
region, but many agencies that 
express interest have not been able to 
get involved in hands-on enrollment 
work. We were interested in learning 
whether a stipend to offset some 
costs and provide an incentive for this 
additional work might prompt some 
groups to get more involved. We 



“There is a 
lot of interest in 
children’s health 
insurance in this region, 
but many agencies that 
express interest have not been 
able to get invoh^ed in hands- 
on enrollment work. We were 
interested in learning whether 
a stipend to offset some costs 
and provide an incentive for 
this additional work might 
prompt some groups 
to get more involved.” 

- Pat Redmond, PCCY 
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designed the enrollment fund to test 
two closely-related concepts: one, 
whether community groups can and 
will integrate enrollment work into 
their ongoing operations without the 
support of dedicated staff, and two, 
whether system barriers would inter- 
fere either with the groups’ willing- 
ness to do this work or with their 
success in enrolling children. These 
were, in effect, our outreach strategy 
research questions and all of our 
planning was done with these under- 
lying questions in mind. 



NACA: Can you be more concrete? 

Redmond: Sure. Since we knew what 
we wanted to learn, we also knew that 
we had to identify the specific pieces 
of information we’d need to have to 
get those answers, and then build in 
a way to gather it. This meant we had 
to consider what the participating 
organizations would do, what infor- 
mation and resources they would 
have to have, and how we would pro- 
vide that to them “ that is, we had 
to begin developing our training and 
monitoring strategies right up front. 



NACA: Let’s talk about 
Infdrmation - what Information 
did you decide you’d need In order 
to find out whether the enrollment 
fund strategy was working? 

Redmond: Simple things, really. We 
needed to know how many applica- 
tions were being submitted as a result 
of the fund, and how many were 
being approved. We also needed to 
know what problems agencies were 
experiencing, and have a mechanism 
for identifying whether they were 
agency problems or problems caused 
by the larger system. 



In consultation with an advisory com- 
mittee, we selected ten agencies with a 
total of 31 sites. We offered two in- 
depth training sessions with the partic- 
ipating agencies. We established a 
reporting and tracking process to give 
us the information we needed to moni- 
tor applications and identify potential 
problems. Participating agencies keep 
copies of all applications and support- 
ing documentation, which they submit 
directly - note that in Pennsylvania, 
the insurance companies process CHIP 
applications, while the state handles 
Medicaid. Every few months, the agen- 
cies send PCCY what we call a tracking 
form, which contains demographics 
and other key information on each 
application they have turned 
in. We use the tracking 
forms to monitor the 
numbers generated by 
each participating 
agency. This informa- 
tion also determines 
the enrollment fund 
reimbursements for 
each agency. 



We also use the 
tracking forms to 
select a random 
sample of the applica- 
tions, and to identify 
how many were 
accepted into CHIP or 
Medicaid, respectively. 

We are fortunate in that we 
were able to get the assistance 
of a professional evaluator who 
agreed to select the random sample 
and track the results, pro bono. We’re 
in the process right now of getting 
our first reports back, where we’ll 
learn how many of the applications 
that were submitted were approved, 
rejected, or are pending. 



”We will use 
the data on approval 
rates to help identify 
and address possible 
problems. If the approval 
rate of applications submitted 
by a given agenc>' falls below 
80 %, we will review the cases 
and discuss the situation with 
the agency. This is one way 
in which we Ve built ongoing 
evaluation and opportunities 
for course corrections 
into our work.“ 



NACA: Are you counting 
completed, submitted, or 
approved applications? 

Redmond: Our advisory committee 
decided early on that each organiza- 
tion would complete an application 
tracking form, and that the form 
would track not application forms 
given out, but each application that 
was completed and submitted. That 
early decision reflected our emphasis 
on achieving actual enrollment, not 
simply broad distribution of the 
enrollment forms. 

It also became clear quite quickly 
that documentation of approval of 
applications would be tough. 
Finding out what happens 
after applications are com- 
pleted and submitted is 



time-consuming and 
resource intensive. We 
considered tracking 
approved applications 
and paying the $25 
enrollment fund fee 
only for those. But 
we needed to keep 
the needs of the 
participating organi- 
zations in mind as 
well as our own 
resources and priori- 
ties. We decided that 



- Pat R£dmond, PCCY 



tying payments to 
approved applications 
alone would have made 
the work more risky and less 
attractive to the participating organi- 
zations. As just noted, we are track- 
ing acceptance for a sample of appli- 
cations because we also need to 
know whether we’re achieving the 
ultimate goal of the program ” 
enrolling kids in CHIP or Medicaid. 
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continued from inside 

eiso 

That gets to the specific 
objective of the enrollment fund proj- 
ect “ to see if community groups can 
integrate enrollment work into their 
ongoing operations successfully. We 
v^ll use the data on approval rates to 
help identify and address possible 
problems. If the approval rate of appli- 
cations submitted by a given agency 
falls below 80%, we will review the 
cases and discuss the situation with 
the agency. This is one way in which 
we’ve built ongoing evaluation and 
opportunities for course corrections 
into our work. There may be issues 
with that organization; if so, we will 
address them. But there may be sys- 
temic issues related to how the applica- 
tions are processed, issues indicating 
that some advocacy with the state is 
needed. If the rates are low for just one 
organization, it is likely that the issue 
lies with the organization. But if 
approval rates are low for many organi- 
zations, it is likely to be a state system 
issue. Of course, we also need to moni- 
tor our work to make sure applications 
are submitted for families who are 
screened and are likely to be eligible. 

NACAs Let’s get bm^k t® tlie pmir 
eet pmeessp Hew 4M ym 

identify the partieipating greyps? 

Redmonds Our Greater Philadelphia 
Child Health Insurance Coalition 
mailing list provided our starting 
point. Because we wanted to get start- 
ed quickly, we decided to use only the 
coalition’s existing mailing list of 
organizations to gauge interest. 
Twenty-two were interested. 



O 




With the assistance of our advisory 
committee, we selected 10 organiza- 
tions for participation. Organizations 
had filled out a simple form for us, 
telling us what groups they expected 
to target and how the Enrollment 
Fund would work in their agency. Let 
me mention that we could have select- 
ed all 22, and maybe recruited more. 
And we would gladly have done this, 
but needed to limit the number for 
administrative reasons. We selected 
agencies which would provide the 
project with geographic and demo- 
graphic diversity, as well as agencies 
where enrollment work seemed espe- 
cially urgent. The participating groups 
include a YWCA, a community center. 
Catholic Social Services, a consortium 
of nurse-managed health centers, and 
a mobile health van, among others. 

M&w giiii ym pr©p@r® 
th® fm th® 

®iar©ilisti©0s^ feart wffkf 

Reiimesids We provided two sets of 
intensive training for the participating 
organizations. In general, each organi- 
zation had identified one or more staff 
per site who were likely to be doing 
the outreach/ enrollment work, and 
these were the staff who participated 
in the trainings. The first training 
described Pennsylvania’s CHIP and 
Medicaid programs, how they are 
structured, how they interact, and 
what kinds of questions and informa- 
tion needs the staff were likely to 
encounter. The second was a hands-on 
training, with forms, paper, and pen- 
cil. The staff practiced enrolling, using 
a variety of case studies. They were 
trained in completing both the CHIP 
and the Medicaid application forms. 
Since Pennsylvania still uses two 
forms, and since the CHIP application 
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is one page and the Medicaid applica- 
tion form is six pages long, there were 
lots of questions. We focused on the 
particulars of how to complete and 
send in both forms, with particular 
attention to the demanding matter of 
income documentation. 

Yoy’r® traeking b®th th® 
v®iysfi® ©f appBIcattoiio 

wMteB* a sampl© of ^liosii 
hm® aioe@pt@do Ar© ym mMg 
my otiiosr gayg© of kow well 
tOs© ps'ooose Is worSdesg for tk© 
partiolpatiBig 

Ro^ifnom^s We built in several ways of 
evaluating whether things are going 
well and where more resources or pro- 
gram changes may be needed. Early on, 
we offered additional training to organi- 
zations when questions and comments 
indicated a need. About two months 
into the work, we did a set of phone 
interviews with the organizations. 

These interviews uncovered some real 
enthusiasm and some frustrations as 
well. It seems to take three to four con- 
tacts with a family to complete an 
application, and the amount of the 
enrollment fund fee - we’re paying $25 
- is not equal to the work they are con- 
tributing. On the other hand, as non 
profits, their motivation is to help. 
Interestingly, initial results seem to indi- 
cate that those organizations new to 
the issue were among the most ener- 
gized with the highest numbers of 
applications turned in. The question of 
how long the enthusiasm can be sus- 
tained is a good one, one which is at 
the heart of the enrollment fund proj- 
ect. We are having an advisory commit- 
tee meeting soon in which we will look 
at how the process is working, how/if it 
should be modified, and how to help 
those organizations that are struggling. 
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NACA: Do you have any data on numbers 
of completed applications to date? 

Redmond: In our first three months, 187 applications were 
submitted, and 20 were still being processed within the organ- 
izations. Additionally, a mobile health van lost all of its com- 
pleted applications when it was caught in a flood during 
Hurricane Floyd, so this number is low. We had anticipated 
800 applications over a year, so this shows that we’re on track. 

NACA: What's next? 

Redmond: In the short run, more data, more program 
refinements, and more evaluation. If the enrollment fund 
concept succeeds as we have defined success, we plan on 
continuing it lor the three-year duration of the CoveringKids 
initiative, at which point funding will cease. We need anoth- 
er several months worth of data on results before we will 
know. Once the program is solidly launched, participating 
organizations will turn applications in to PCCY only once 
every six months. We are pleased that organizations that 
applied but were not chosen as direct participants are still 
involved through help with outreach. 

iVACA thanks PCCY and Pat Redmond for sharing informa- 
tion on this carefully planned and well-thought out outreach 
and enrollment strategy. It shows that “Planning the 
Beginning with the End in Mind” - that is, keeping a clear 
focus on the purpose of the project and what you want to learn 
at the time the project is first designed - can create a program 
which can be evaluated for effectiveness, not just at the end, 
but while the project is underway. Every project component - 
planning the activities, identi^nng the participants, develop- 
ing the training, deciding on the data to be collected, and 
designing the tracking forms, or more - needs to be measured 
against these questions: why are we doing this project, and 
what do we want to know when the project is over. PCCY has 
developed a project design which not only meets this standard, 
but which will give them the opportunity for course corrections 
early in the project The need for effective outreach and enroll- 
ment strategies is urgent The need to know whether the strate- 
gies we attempt are working is equally urgent for child advo- 
cates to help identify and promote strategies that work. The 
PCCY example shows that it can be done. 
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1. In a reflection of the extent of state control over the SCHIP program, the 
names of such programs vary widely, sometimes even county by county. 

For purposes of this Issue Brief, however, except when discussing 
Pennsylvania's SCHIP program, entided Child Health Insurance Program 
or CHIP, all programs will be referred to by the generic term “SCHIP" 

2. For examples, see 1998 State Children 's Health Insurance Program Annual 
Report, published jointly by the National Conference of State Legislatures 
and the National Governors’ Association; The Children‘s Health Insurance 
Program » States’ Application and Enrollment Processes: An Early Report from 
the Front Lines, Office of Inspector General, Department of Health and 
Human Services, OEI-05-98^0310, May 1999; Children’s Health Insurance 
Program - State Implementation Approaches Are Evolving General Accounting 
Office, GAO-HEHS-99-65, May 1999; or “CHIP Outreach and Enrollmenc 
A View from the States," American Public Human Services Association, 
September 1999. 

3. See www.insurekidsnow.gov for the web-based component of this 
campaign. 

4. See www.CoveringKids.org for an overview of the Robert Wood Johnson 
Foundation- funded CoveringKids initiative, its grantees and programs in 
the states, and for links with additional information about the program and 
its objectives. 

5. Enrollment Inaeases in State CHIP Programs: December 1998 to June 1999, 
Vernon K. Smith, Health Management Associates, for The Kaiser 
Commission on Medicaid and the Uninsured, July 30, 1999. 

6. The State Children’s Health Insurance Program Annual Enrollment Report, 
October I, 1998 through September 30, 1999 based on reports from 35 states 
and projections for others. Note that these two sets of data measure differ- 
ent things - the first measures enrollment as of a given date, as defined by 
state receipt of Title XXI matching funds; the second measures total undu- 
plicated enrollment over the period of the federal fiscal year. The total 
enrollment over the fiscal year is higher than would be a report of total 
enrollment as of, for example, the final day of the federal fiscal year. 

7. See Losing Health Insurance: The Unintended Consequences oj Welfare Reform, 
Families USA, May 1999; The Medicaid Maze: Coverage Expands, But 
Enrollment Problems Persist, Marilyn Ellwood, Mathematica Policy Research, 
Inc., for The Kaiser Commission on Medicaid and the Uninsured, 
September 1999; Medicaid Enrollment: Amid Declines, State Efforts to Ensure 
Coverage After Welfare Reform Vary, General Accounting Office, GAO/HEHS- 
99-163, September 1999; Missed Opportunities: Declining Medicaid 
Enrollment Undermines the Nation's Progress in Insuring Low-Income Children, 
Center on Budget and Policy Priorities, October 1999. 
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